Configure folder scheduler in YAML
This commit is contained in:
parent
b61ab0604a
commit
7fd92a72df
|
|
@ -58,21 +58,24 @@ Sources:
|
||||||
|
|
||||||
Add a new scheduler config group under `video-clipping.folder-scheduler`.
|
Add a new scheduler config group under `video-clipping.folder-scheduler`.
|
||||||
|
|
||||||
Recommended properties:
|
Recommended YAML:
|
||||||
|
|
||||||
```properties
|
```yaml
|
||||||
video-clipping.folder-scheduler.enabled=true
|
video-clipping:
|
||||||
video-clipping.folder-scheduler.input-directory=/input/source
|
folder-scheduler:
|
||||||
video-clipping.folder-scheduler.output-directory=/output/clips
|
enabled: true
|
||||||
video-clipping.folder-scheduler.processed-directory=/input/processed
|
input-directory: /input/source
|
||||||
video-clipping.folder-scheduler.rejected-directory=/input/rejected
|
output-directory: /output/clips
|
||||||
video-clipping.folder-scheduler.working-directory=/input/working
|
processed-directory: /input/processed
|
||||||
video-clipping.folder-scheduler.poll-interval-ms=5000
|
rejected-directory: /input/rejected
|
||||||
video-clipping.folder-scheduler.segment-duration-seconds=8
|
working-directory: /input/working
|
||||||
video-clipping.folder-scheduler.ffmpeg-binary=ffmpeg
|
poll-interval-ms: 5000
|
||||||
video-clipping.folder-scheduler.ffprobe-binary=ffprobe
|
segment-duration-seconds: 8
|
||||||
video-clipping.folder-scheduler.output-container=mp4
|
ffmpeg-binary: ffmpeg
|
||||||
video-clipping.folder-scheduler.exact-preset=veryfast
|
ffprobe-binary: ffprobe
|
||||||
|
output-container: mp4
|
||||||
|
exact-preset: veryfast
|
||||||
|
preserve-input-quality: true
|
||||||
```
|
```
|
||||||
|
|
||||||
Keep the existing API-driven clipper config separate from this folder scheduler. The folder scheduler is a direct filesystem workflow, not an API/upload/session workflow.
|
Keep the existing API-driven clipper config separate from this folder scheduler. The folder scheduler is a direct filesystem workflow, not an API/upload/session workflow.
|
||||||
|
|
@ -173,35 +176,29 @@ Do not log raw paths from untrusted metadata or full FFmpeg stderr if it contain
|
||||||
|
|
||||||
## FFmpeg Command
|
## FFmpeg Command
|
||||||
|
|
||||||
Default to exact 8-second MP4/H.264/AAC output:
|
Default to quality-preserving MP4 stream copy when the source already has usable keyframes:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
ffmpeg -hide_banner -y \
|
ffmpeg -hide_banner -y \
|
||||||
-i /input/working/1.mp4 \
|
-i /input/working/1.mp4 \
|
||||||
-map 0 \
|
-map 0 -c copy \
|
||||||
-c:v libx264 \
|
|
||||||
-preset veryfast \
|
|
||||||
-crf 20 \
|
|
||||||
-force_key_frames "expr:gte(t,n_forced*8)" \
|
|
||||||
-c:a aac \
|
|
||||||
-b:a 128k \
|
|
||||||
-f segment \
|
-f segment \
|
||||||
-segment_time 8 \
|
-segment_time 8 \
|
||||||
-segment_format_options movflags=+faststart \
|
|
||||||
-reset_timestamps 1 \
|
-reset_timestamps 1 \
|
||||||
/output/clips/1/clip_%05d.mp4
|
/output/clips/1/clip_%05d.mp4
|
||||||
```
|
```
|
||||||
|
|
||||||
Expected behavior:
|
Expected behavior:
|
||||||
|
|
||||||
- each generated clip is approximately 8 seconds
|
- each generated clip preserves the input codec quality because no re-encode occurs
|
||||||
|
- clips are cut on keyframes, so boundaries may drift from exact 8-second marks
|
||||||
- the final clip may be shorter than 8 seconds
|
- the final clip may be shorter than 8 seconds
|
||||||
- output files are broadly playable MP4 files
|
- output files are broadly playable MP4 files
|
||||||
|
|
||||||
Implementation note:
|
Implementation note:
|
||||||
|
|
||||||
- If exact boundaries are more important than speed, always transcode with `libx264` and force keyframes.
|
- If exact 8-second boundaries are more important than preserving source quality, transcode with `libx264` and force keyframes.
|
||||||
- If speed is more important and keyframe-aligned cuts are acceptable, add an optional `FAST` mode using `-c copy`, but document that clips may not start exactly at 8-second boundaries.
|
- If preserving source quality is more important, use stream copy and accept keyframe-aligned cuts.
|
||||||
|
|
||||||
## Proposed Code Structure
|
## Proposed Code Structure
|
||||||
|
|
||||||
|
|
@ -268,6 +265,19 @@ This ensures the scheduler never processes the same source file repeatedly.
|
||||||
|
|
||||||
The scheduler uses an initial delay and fixed delay of `5000` ms by default. The initial delay lets startup directory initialization finish; subsequent delays start after the previous scan finishes, so scans never intentionally overlap. Configure both with `video-clipping.folder-scheduler.poll-interval-ms` (minimum `1000` ms).
|
The scheduler uses an initial delay and fixed delay of `5000` ms by default. The initial delay lets startup directory initialization finish; subsequent delays start after the previous scan finishes, so scans never intentionally overlap. Configure both with `video-clipping.folder-scheduler.poll-interval-ms` (minimum `1000` ms).
|
||||||
|
|
||||||
|
The base configuration enables repository-local folder processing for normal IDE starts. Production deployments can override it without changing the artifact:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
FOLDER_SCHEDULER_ENABLED=true
|
||||||
|
FOLDER_SCHEDULER_INPUT_DIRECTORY=/input/source
|
||||||
|
FOLDER_SCHEDULER_OUTPUT_DIRECTORY=/output/clips
|
||||||
|
FOLDER_SCHEDULER_WORKING_DIRECTORY=/input/working
|
||||||
|
FOLDER_SCHEDULER_PROCESSED_DIRECTORY=/input/processed
|
||||||
|
FOLDER_SCHEDULER_REJECTED_DIRECTORY=/input/rejected
|
||||||
|
FOLDER_SCHEDULER_POLL_INTERVAL_MS=5000
|
||||||
|
FOLDER_SCHEDULER_LOG_LEVEL=INFO
|
||||||
|
```
|
||||||
|
|
||||||
Lifecycle logs use stable `event=...` fields and a per-run `scan_id` for correlation. Active work and state transitions log at info or warn/error level; scan start, idle, completion, directory readiness, and probe timing log at debug level.
|
Lifecycle logs use stable `event=...` fields and a per-run `scan_id` for correlation. Active work and state transitions log at info or warn/error level; scan start, idle, completion, directory readiness, and probe timing log at debug level.
|
||||||
|
|
||||||
Log these events:
|
Log these events:
|
||||||
|
|
|
||||||
9
pom.xml
9
pom.xml
|
|
@ -87,6 +87,15 @@
|
||||||
|
|
||||||
<build>
|
<build>
|
||||||
<plugins>
|
<plugins>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
|
<artifactId>maven-surefire-plugin</artifactId>
|
||||||
|
<configuration>
|
||||||
|
<systemPropertyVariables>
|
||||||
|
<spring.profiles.active>test</spring.profiles.active>
|
||||||
|
</systemPropertyVariables>
|
||||||
|
</configuration>
|
||||||
|
</plugin>
|
||||||
<plugin>
|
<plugin>
|
||||||
<groupId>org.springframework.boot</groupId>
|
<groupId>org.springframework.boot</groupId>
|
||||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||||
|
|
|
||||||
|
|
@ -368,6 +368,8 @@ public class VideoClippingProperties {
|
||||||
|
|
||||||
private String exactPreset = "veryfast";
|
private String exactPreset = "veryfast";
|
||||||
|
|
||||||
|
private boolean preserveInputQuality = true;
|
||||||
|
|
||||||
public boolean isEnabled() {
|
public boolean isEnabled() {
|
||||||
return enabled;
|
return enabled;
|
||||||
}
|
}
|
||||||
|
|
@ -463,6 +465,14 @@ public class VideoClippingProperties {
|
||||||
public void setExactPreset(String exactPreset) {
|
public void setExactPreset(String exactPreset) {
|
||||||
this.exactPreset = exactPreset;
|
this.exactPreset = exactPreset;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public boolean isPreserveInputQuality() {
|
||||||
|
return preserveInputQuality;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setPreserveInputQuality(boolean preserveInputQuality) {
|
||||||
|
this.preserveInputQuality = preserveInputQuality;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
public static class Quotas {
|
public static class Quotas {
|
||||||
|
|
|
||||||
|
|
@ -38,30 +38,13 @@ public class FolderFfmpegClipper {
|
||||||
Path outputDirectory = createOutputDirectory(sourceFile);
|
Path outputDirectory = createOutputDirectory(sourceFile);
|
||||||
String segmentDuration = Integer.toString(properties.getSegmentDurationSeconds());
|
String segmentDuration = Integer.toString(properties.getSegmentDurationSeconds());
|
||||||
Path outputPattern = outputDirectory.resolve("clip_%05d." + properties.getOutputContainer());
|
Path outputPattern = outputDirectory.resolve("clip_%05d." + properties.getOutputContainer());
|
||||||
List<String> command = List.of(
|
List<String> command = buildCommand(sourceFile, segmentDuration, outputPattern);
|
||||||
properties.getFfmpegBinary(),
|
|
||||||
"-hide_banner", "-y",
|
|
||||||
"-i", sourceFile.toString(),
|
|
||||||
"-map", "0:v:0",
|
|
||||||
"-map", "0:a?",
|
|
||||||
"-c:v", "libx264",
|
|
||||||
"-preset", properties.getExactPreset(),
|
|
||||||
"-crf", "20",
|
|
||||||
"-force_key_frames", "expr:gte(t,n_forced*" + segmentDuration + ")",
|
|
||||||
"-c:a", "aac",
|
|
||||||
"-b:a", "128k",
|
|
||||||
"-f", "segment",
|
|
||||||
"-segment_time", segmentDuration,
|
|
||||||
"-segment_format_options", "movflags=+faststart",
|
|
||||||
"-reset_timestamps", "1",
|
|
||||||
outputPattern.toString()
|
|
||||||
);
|
|
||||||
|
|
||||||
ProcessResult processResult;
|
ProcessResult processResult;
|
||||||
long startedAt = System.nanoTime();
|
long startedAt = System.nanoTime();
|
||||||
log.info("event=ffmpeg_started file={} output_directory={} segment_duration_seconds={} preset={}",
|
log.info("event=ffmpeg_started file={} output_directory={} segment_duration_seconds={} quality_mode={} preset={}",
|
||||||
sourceFile.getFileName(), outputDirectory, properties.getSegmentDurationSeconds(),
|
sourceFile.getFileName(), outputDirectory, properties.getSegmentDurationSeconds(),
|
||||||
properties.getExactPreset());
|
qualityMode(), properties.getExactPreset());
|
||||||
try {
|
try {
|
||||||
processResult = processExecutor.execute(command);
|
processResult = processExecutor.execute(command);
|
||||||
} catch (InterruptedException ex) {
|
} catch (InterruptedException ex) {
|
||||||
|
|
@ -83,6 +66,53 @@ public class FolderFfmpegClipper {
|
||||||
return new ClipResult(outputDirectory, clipCount);
|
return new ClipResult(outputDirectory, clipCount);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private List<String> buildCommand(Path sourceFile, String segmentDuration, Path outputPattern) {
|
||||||
|
List<String> command = new java.util.ArrayList<>();
|
||||||
|
command.add(properties.getFfmpegBinary());
|
||||||
|
command.add("-hide_banner");
|
||||||
|
command.add("-y");
|
||||||
|
command.add("-i");
|
||||||
|
command.add(sourceFile.toString());
|
||||||
|
command.add("-map");
|
||||||
|
command.add("0");
|
||||||
|
if (properties.isPreserveInputQuality()) {
|
||||||
|
command.add("-c");
|
||||||
|
command.add("copy");
|
||||||
|
} else {
|
||||||
|
command.add("-c:v");
|
||||||
|
command.add("libx264");
|
||||||
|
command.add("-preset");
|
||||||
|
command.add(properties.getExactPreset());
|
||||||
|
command.add("-crf");
|
||||||
|
command.add("20");
|
||||||
|
command.add("-force_key_frames");
|
||||||
|
command.add("expr:gte(t,n_forced*" + segmentDuration + ")");
|
||||||
|
command.add("-c:a");
|
||||||
|
command.add("aac");
|
||||||
|
command.add("-b:a");
|
||||||
|
command.add("128k");
|
||||||
|
}
|
||||||
|
command.add("-f");
|
||||||
|
command.add("segment");
|
||||||
|
command.add("-segment_time");
|
||||||
|
command.add(segmentDuration);
|
||||||
|
if (properties.isPreserveInputQuality()) {
|
||||||
|
command.add("-reset_timestamps");
|
||||||
|
command.add("1");
|
||||||
|
} else {
|
||||||
|
command.add("-segment_format_options");
|
||||||
|
command.add("movflags=+faststart");
|
||||||
|
command.add("-reset_timestamps");
|
||||||
|
command.add("1");
|
||||||
|
}
|
||||||
|
command.add(outputPattern.toString());
|
||||||
|
return command;
|
||||||
|
}
|
||||||
|
|
||||||
|
private String qualityMode() {
|
||||||
|
return properties.isPreserveInputQuality() ? "copy" : "transcode";
|
||||||
|
}
|
||||||
|
|
||||||
private long elapsedMillis(long startedAt) {
|
private long elapsedMillis(long startedAt) {
|
||||||
return TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startedAt);
|
return TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startedAt);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -4,6 +4,8 @@ import org.example.videoclips.config.VideoClippingProperties;
|
||||||
import org.slf4j.Logger;
|
import org.slf4j.Logger;
|
||||||
import org.slf4j.LoggerFactory;
|
import org.slf4j.LoggerFactory;
|
||||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
|
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
|
||||||
|
import org.springframework.boot.context.event.ApplicationReadyEvent;
|
||||||
|
import org.springframework.context.event.EventListener;
|
||||||
import org.springframework.scheduling.annotation.Scheduled;
|
import org.springframework.scheduling.annotation.Scheduled;
|
||||||
import org.springframework.stereotype.Component;
|
import org.springframework.stereotype.Component;
|
||||||
|
|
||||||
|
|
@ -41,6 +43,15 @@ public class FolderVideoScanScheduler {
|
||||||
this.clipper = clipper;
|
this.clipper = clipper;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@EventListener(ApplicationReadyEvent.class)
|
||||||
|
public void logStartup() {
|
||||||
|
log.info("event=folder_scheduler_started enabled=true initial_delay_ms={} fixed_delay_ms={} "
|
||||||
|
+ "segment_duration_seconds={} input_directory={} output_directory={} quality_mode={}",
|
||||||
|
properties.getPollIntervalMs(), properties.getPollIntervalMs(),
|
||||||
|
properties.getSegmentDurationSeconds(), properties.getInputDirectory(),
|
||||||
|
properties.getOutputDirectory(), properties.isPreserveInputQuality() ? "copy" : "transcode");
|
||||||
|
}
|
||||||
|
|
||||||
@Scheduled(
|
@Scheduled(
|
||||||
initialDelayString = "${video-clipping.folder-scheduler.poll-interval-ms:5000}",
|
initialDelayString = "${video-clipping.folder-scheduler.poll-interval-ms:5000}",
|
||||||
fixedDelayString = "${video-clipping.folder-scheduler.poll-interval-ms:5000}"
|
fixedDelayString = "${video-clipping.folder-scheduler.poll-interval-ms:5000}"
|
||||||
|
|
@ -115,9 +126,15 @@ public class FolderVideoScanScheduler {
|
||||||
FolderVideoValidator.ValidationResult result = validator.validate(workingFile);
|
FolderVideoValidator.ValidationResult result = validator.validate(workingFile);
|
||||||
if (result.valid()) {
|
if (result.valid()) {
|
||||||
log.info("event=validation_succeeded scan_id={} file={}", scanId, workingFile.getFileName());
|
log.info("event=validation_succeeded scan_id={} file={}", scanId, workingFile.getFileName());
|
||||||
|
long clippingStartedAt = System.nanoTime();
|
||||||
|
log.info("event=video_processing_started scan_id={} file={} segment_duration_seconds={} quality_mode={}",
|
||||||
|
scanId, workingFile.getFileName(), properties.getSegmentDurationSeconds(),
|
||||||
|
properties.isPreserveInputQuality() ? "copy" : "transcode");
|
||||||
FolderFfmpegClipper.ClipResult clipResult = clipper.clip(workingFile);
|
FolderFfmpegClipper.ClipResult clipResult = clipper.clip(workingFile);
|
||||||
log.info("event=clipping_succeeded scan_id={} file={} clip_count={} output_directory={}",
|
log.info("event=video_processing_completed scan_id={} file={} elapsed_ms={} clip_count={} "
|
||||||
scanId, workingFile.getFileName(), clipResult.clipCount(), clipResult.outputDirectory());
|
+ "clips_folder_name={} clips_folder={}",
|
||||||
|
scanId, workingFile.getFileName(), elapsedMillis(clippingStartedAt), clipResult.clipCount(),
|
||||||
|
clipResult.outputDirectory().getFileName(), clipResult.outputDirectory());
|
||||||
Path processedFile = moveToDirectory(workingFile, Path.of(properties.getProcessedDirectory()));
|
Path processedFile = moveToDirectory(workingFile, Path.of(properties.getProcessedDirectory()));
|
||||||
log.info("event=processing_succeeded scan_id={} file={} processed_directory={}",
|
log.info("event=processing_succeeded scan_id={} file={} processed_directory={}",
|
||||||
scanId, processedFile.getFileName(), processedFile.getParent());
|
scanId, processedFile.getFileName(), processedFile.getParent());
|
||||||
|
|
|
||||||
|
|
@ -1,13 +0,0 @@
|
||||||
video-clipping.folder-scheduler.enabled=true
|
|
||||||
video-clipping.folder-scheduler.input-directory=./input/source
|
|
||||||
video-clipping.folder-scheduler.output-directory=./output/clips
|
|
||||||
video-clipping.folder-scheduler.processed-directory=./input/processed
|
|
||||||
video-clipping.folder-scheduler.rejected-directory=./input/rejected
|
|
||||||
video-clipping.folder-scheduler.working-directory=./input/working
|
|
||||||
video-clipping.folder-scheduler.poll-interval-ms=5000
|
|
||||||
video-clipping.folder-scheduler.segment-duration-seconds=8
|
|
||||||
video-clipping.folder-scheduler.ffmpeg-binary=ffmpeg
|
|
||||||
video-clipping.folder-scheduler.ffprobe-binary=ffprobe
|
|
||||||
video-clipping.folder-scheduler.output-container=mp4
|
|
||||||
video-clipping.folder-scheduler.exact-preset=veryfast
|
|
||||||
logging.level.org.example.videoclips.folder=DEBUG
|
|
||||||
|
|
@ -0,0 +1,19 @@
|
||||||
|
video-clipping:
|
||||||
|
folder-scheduler:
|
||||||
|
enabled: true
|
||||||
|
input-directory: ./input/source
|
||||||
|
output-directory: ./output/clips
|
||||||
|
processed-directory: ./input/processed
|
||||||
|
rejected-directory: ./input/rejected
|
||||||
|
working-directory: ./input/working
|
||||||
|
poll-interval-ms: 5000
|
||||||
|
segment-duration-seconds: 8
|
||||||
|
ffmpeg-binary: ffmpeg
|
||||||
|
ffprobe-binary: ffprobe
|
||||||
|
output-container: mp4
|
||||||
|
exact-preset: veryfast
|
||||||
|
preserve-input-quality: true
|
||||||
|
|
||||||
|
logging:
|
||||||
|
level:
|
||||||
|
org.example.videoclips.folder: INFO
|
||||||
|
|
@ -25,18 +25,6 @@ video-clipping.ffmpeg.ffmpeg-binary=ffmpeg
|
||||||
video-clipping.ffmpeg.input-directory=./tmp/ffmpeg-input
|
video-clipping.ffmpeg.input-directory=./tmp/ffmpeg-input
|
||||||
video-clipping.ffmpeg.output-directory=./tmp/ffmpeg-output
|
video-clipping.ffmpeg.output-directory=./tmp/ffmpeg-output
|
||||||
video-clipping.ffmpeg.cleanup-local-files=true
|
video-clipping.ffmpeg.cleanup-local-files=true
|
||||||
video-clipping.folder-scheduler.enabled=false
|
|
||||||
video-clipping.folder-scheduler.input-directory=/input/source
|
|
||||||
video-clipping.folder-scheduler.output-directory=/output/clips
|
|
||||||
video-clipping.folder-scheduler.processed-directory=/input/processed
|
|
||||||
video-clipping.folder-scheduler.rejected-directory=/input/rejected
|
|
||||||
video-clipping.folder-scheduler.working-directory=/input/working
|
|
||||||
video-clipping.folder-scheduler.poll-interval-ms=5000
|
|
||||||
video-clipping.folder-scheduler.segment-duration-seconds=8
|
|
||||||
video-clipping.folder-scheduler.ffmpeg-binary=ffmpeg
|
|
||||||
video-clipping.folder-scheduler.ffprobe-binary=ffprobe
|
|
||||||
video-clipping.folder-scheduler.output-container=mp4
|
|
||||||
video-clipping.folder-scheduler.exact-preset=veryfast
|
|
||||||
video-clipping.quotas.max-active-jobs-per-tenant=5
|
video-clipping.quotas.max-active-jobs-per-tenant=5
|
||||||
video-clipping.cleanup.enabled=true
|
video-clipping.cleanup.enabled=true
|
||||||
video-clipping.cleanup.local-artifact-poll-interval-ms=300000
|
video-clipping.cleanup.local-artifact-poll-interval-ms=300000
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,19 @@
|
||||||
|
video-clipping:
|
||||||
|
folder-scheduler:
|
||||||
|
enabled: ${FOLDER_SCHEDULER_ENABLED:true}
|
||||||
|
input-directory: ${FOLDER_SCHEDULER_INPUT_DIRECTORY:./input/source}
|
||||||
|
output-directory: ${FOLDER_SCHEDULER_OUTPUT_DIRECTORY:./output/clips}
|
||||||
|
processed-directory: ${FOLDER_SCHEDULER_PROCESSED_DIRECTORY:./input/processed}
|
||||||
|
rejected-directory: ${FOLDER_SCHEDULER_REJECTED_DIRECTORY:./input/rejected}
|
||||||
|
working-directory: ${FOLDER_SCHEDULER_WORKING_DIRECTORY:./input/working}
|
||||||
|
poll-interval-ms: ${FOLDER_SCHEDULER_POLL_INTERVAL_MS:5000}
|
||||||
|
segment-duration-seconds: 8
|
||||||
|
ffmpeg-binary: ffmpeg
|
||||||
|
ffprobe-binary: ffprobe
|
||||||
|
output-container: mp4
|
||||||
|
exact-preset: veryfast
|
||||||
|
preserve-input-quality: ${FOLDER_SCHEDULER_PRESERVE_INPUT_QUALITY:true}
|
||||||
|
|
||||||
|
logging:
|
||||||
|
level:
|
||||||
|
org.example.videoclips.folder: ${FOLDER_SCHEDULER_LOG_LEVEL:INFO}
|
||||||
|
|
@ -1,73 +0,0 @@
|
||||||
package org.example.videoclips.api;
|
|
||||||
|
|
||||||
import org.junit.jupiter.api.Test;
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
|
||||||
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
|
|
||||||
import org.springframework.boot.test.context.SpringBootTest;
|
|
||||||
import org.springframework.http.MediaType;
|
|
||||||
import org.springframework.test.web.servlet.MockMvc;
|
|
||||||
|
|
||||||
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post;
|
|
||||||
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
|
|
||||||
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
|
|
||||||
|
|
||||||
@SpringBootTest(properties = "video-clipping.quotas.max-active-jobs-per-tenant=0")
|
|
||||||
@AutoConfigureMockMvc
|
|
||||||
class TenantQuotaControllerTest {
|
|
||||||
|
|
||||||
@Autowired
|
|
||||||
private MockMvc mockMvc;
|
|
||||||
|
|
||||||
@Test
|
|
||||||
void rejectsClipJobCreationWhenTenantActiveJobQuotaIsExceeded() throws Exception {
|
|
||||||
String assetResponse = mockMvc.perform(post("/v1/video-assets")
|
|
||||||
.header("X-Tenant-Id", "tenant-a")
|
|
||||||
.contentType(MediaType.APPLICATION_JSON)
|
|
||||||
.content("""
|
|
||||||
{
|
|
||||||
"fileName": "quota.mp4",
|
|
||||||
"contentType": "video/mp4",
|
|
||||||
"contentLengthBytes": 4096,
|
|
||||||
"checksumSha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
|
|
||||||
}
|
|
||||||
"""))
|
|
||||||
.andExpect(status().isCreated())
|
|
||||||
.andReturn()
|
|
||||||
.getResponse()
|
|
||||||
.getContentAsString();
|
|
||||||
|
|
||||||
String assetId = JsonTestHelper.read(assetResponse, "$.assetId");
|
|
||||||
String uploadId = JsonTestHelper.read(assetResponse, "$.upload.uploadId");
|
|
||||||
|
|
||||||
mockMvc.perform(post("/v1/video-assets/{assetId}/uploads:complete", assetId)
|
|
||||||
.header("X-Tenant-Id", "tenant-a")
|
|
||||||
.contentType(MediaType.APPLICATION_JSON)
|
|
||||||
.content("""
|
|
||||||
{
|
|
||||||
"uploadId": "%s",
|
|
||||||
"sourceDurationSeconds": 17,
|
|
||||||
"parts": [
|
|
||||||
{
|
|
||||||
"partNumber": 1,
|
|
||||||
"etag": "\\"etag-1\\""
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
""".formatted(uploadId)))
|
|
||||||
.andExpect(status().isOk());
|
|
||||||
|
|
||||||
mockMvc.perform(post("/v1/video-assets/{assetId}/clip-jobs", assetId)
|
|
||||||
.header("X-Tenant-Id", "tenant-a")
|
|
||||||
.contentType(MediaType.APPLICATION_JSON)
|
|
||||||
.content("""
|
|
||||||
{
|
|
||||||
"segmentDurationSeconds": 8,
|
|
||||||
"accuracyMode": "FAST",
|
|
||||||
"outputContainer": "mp4",
|
|
||||||
"overwriteExisting": false
|
|
||||||
}
|
|
||||||
"""))
|
|
||||||
.andExpect(status().isTooManyRequests())
|
|
||||||
.andExpect(jsonPath("$.type").value("https://api.example.com/problems/quota-exceeded"));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -20,7 +20,7 @@ class FolderFfmpegClipperTest {
|
||||||
Path tempDir;
|
Path tempDir;
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
void buildsExactEightSecondMp4CommandAndCountsClips() throws Exception {
|
void buildsQualityPreservingMp4CommandAndCountsClips() throws Exception {
|
||||||
AtomicReference<List<String>> command = new AtomicReference<>();
|
AtomicReference<List<String>> command = new AtomicReference<>();
|
||||||
FolderFfmpegClipper clipper = clipper(arguments -> {
|
FolderFfmpegClipper clipper = clipper(arguments -> {
|
||||||
command.set(arguments);
|
command.set(arguments);
|
||||||
|
|
@ -36,11 +36,28 @@ class FolderFfmpegClipperTest {
|
||||||
|
|
||||||
assertEquals(tempDir.resolve("output/movie"), result.outputDirectory());
|
assertEquals(tempDir.resolve("output/movie"), result.outputDirectory());
|
||||||
assertEquals(2, result.clipCount());
|
assertEquals(2, result.clipCount());
|
||||||
|
assertCommandPair(command.get(), "-c", "copy");
|
||||||
|
assertCommandPair(command.get(), "-segment_time", "8");
|
||||||
|
assertTrue(command.get().getLast().endsWith("clip_%05d.mp4"));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void buildsTranscodingCommandWhenPreserveInputQualityIsDisabled() throws Exception {
|
||||||
|
AtomicReference<List<String>> command = new AtomicReference<>();
|
||||||
|
VideoClippingProperties properties = properties();
|
||||||
|
properties.getFolderScheduler().setPreserveInputQuality(false);
|
||||||
|
FolderFfmpegClipper clipper = new FolderFfmpegClipper(properties, arguments -> {
|
||||||
|
command.set(arguments);
|
||||||
|
Path outputDirectory = Path.of(arguments.getLast()).getParent();
|
||||||
|
Files.writeString(outputDirectory.resolve("clip_00000.mp4"), "clip");
|
||||||
|
return new FolderFfmpegClipper.ProcessResult(0, "ok");
|
||||||
|
});
|
||||||
|
|
||||||
|
clipper.clip(tempDir.resolve("source/movie.mp4"));
|
||||||
|
|
||||||
assertCommandPair(command.get(), "-c:v", "libx264");
|
assertCommandPair(command.get(), "-c:v", "libx264");
|
||||||
assertCommandPair(command.get(), "-c:a", "aac");
|
assertCommandPair(command.get(), "-c:a", "aac");
|
||||||
assertCommandPair(command.get(), "-segment_time", "8");
|
|
||||||
assertCommandPair(command.get(), "-force_key_frames", "expr:gte(t,n_forced*8)");
|
assertCommandPair(command.get(), "-force_key_frames", "expr:gte(t,n_forced*8)");
|
||||||
assertTrue(command.get().getLast().endsWith("clip_%05d.mp4"));
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,7 @@ package org.example.videoclips.folder;
|
||||||
|
|
||||||
import org.junit.jupiter.api.Test;
|
import org.junit.jupiter.api.Test;
|
||||||
import org.springframework.core.io.ClassPathResource;
|
import org.springframework.core.io.ClassPathResource;
|
||||||
import org.springframework.core.io.support.PropertiesLoaderUtils;
|
import org.springframework.beans.factory.config.YamlPropertiesFactoryBean;
|
||||||
|
|
||||||
import java.util.Properties;
|
import java.util.Properties;
|
||||||
|
|
||||||
|
|
@ -12,8 +12,7 @@ class FolderSchedulerLocalProfileTest {
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
void enablesSchedulerWithRepositoryLocalEightSecondPaths() throws Exception {
|
void enablesSchedulerWithRepositoryLocalEightSecondPaths() throws Exception {
|
||||||
Properties properties = PropertiesLoaderUtils.loadProperties(
|
Properties properties = yamlProperties("application-folder-scheduler-local.yml");
|
||||||
new ClassPathResource("application-folder-scheduler-local.properties"));
|
|
||||||
|
|
||||||
assertEquals("true", properties.getProperty("video-clipping.folder-scheduler.enabled"));
|
assertEquals("true", properties.getProperty("video-clipping.folder-scheduler.enabled"));
|
||||||
assertEquals("./input/source", properties.getProperty("video-clipping.folder-scheduler.input-directory"));
|
assertEquals("./input/source", properties.getProperty("video-clipping.folder-scheduler.input-directory"));
|
||||||
|
|
@ -23,6 +22,16 @@ class FolderSchedulerLocalProfileTest {
|
||||||
assertEquals("./input/rejected", properties.getProperty("video-clipping.folder-scheduler.rejected-directory"));
|
assertEquals("./input/rejected", properties.getProperty("video-clipping.folder-scheduler.rejected-directory"));
|
||||||
assertEquals("8", properties.getProperty("video-clipping.folder-scheduler.segment-duration-seconds"));
|
assertEquals("8", properties.getProperty("video-clipping.folder-scheduler.segment-duration-seconds"));
|
||||||
assertEquals("mp4", properties.getProperty("video-clipping.folder-scheduler.output-container"));
|
assertEquals("mp4", properties.getProperty("video-clipping.folder-scheduler.output-container"));
|
||||||
assertEquals("DEBUG", properties.getProperty("logging.level.org.example.videoclips.folder"));
|
assertEquals("INFO", properties.getProperty("logging.level.org.example.videoclips.folder"));
|
||||||
|
}
|
||||||
|
|
||||||
|
private Properties yamlProperties(String classpathResource) {
|
||||||
|
YamlPropertiesFactoryBean factoryBean = new YamlPropertiesFactoryBean();
|
||||||
|
factoryBean.setResources(new ClassPathResource(classpathResource));
|
||||||
|
Properties properties = factoryBean.getObject();
|
||||||
|
if (properties == null) {
|
||||||
|
throw new IllegalStateException("Unable to load " + classpathResource);
|
||||||
|
}
|
||||||
|
return properties;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -88,6 +88,20 @@ class FolderVideoScanSchedulerTest {
|
||||||
assertTrue(((AtomicBoolean) scanningField.get(scheduler)).get());
|
assertTrue(((AtomicBoolean) scanningField.get(scheduler)).get());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void logsSchedulerStartupConfiguration() {
|
||||||
|
scheduler(tempDir).logStartup();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void logsTranscodingQualityModeWhenConfigured() throws Exception {
|
||||||
|
Files.writeString(tempDir.resolve("1.mp4"), "video");
|
||||||
|
FolderVideoScanScheduler scheduler = scheduler(tempDir, acceptedValidator(), successfulClipper(), false);
|
||||||
|
|
||||||
|
scheduler.logStartup();
|
||||||
|
scheduler.scan();
|
||||||
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
void scanReleasesGuardAfterSelectingCandidate() throws Exception {
|
void scanReleasesGuardAfterSelectingCandidate() throws Exception {
|
||||||
Files.writeString(tempDir.resolve("1.mp4"), "video");
|
Files.writeString(tempDir.resolve("1.mp4"), "video");
|
||||||
|
|
@ -257,9 +271,19 @@ class FolderVideoScanSchedulerTest {
|
||||||
Path inputDirectory,
|
Path inputDirectory,
|
||||||
FolderVideoValidator validator,
|
FolderVideoValidator validator,
|
||||||
FolderFfmpegClipper clipper
|
FolderFfmpegClipper clipper
|
||||||
|
) {
|
||||||
|
return scheduler(inputDirectory, validator, clipper, true);
|
||||||
|
}
|
||||||
|
|
||||||
|
private FolderVideoScanScheduler scheduler(
|
||||||
|
Path inputDirectory,
|
||||||
|
FolderVideoValidator validator,
|
||||||
|
FolderFfmpegClipper clipper,
|
||||||
|
boolean preserveInputQuality
|
||||||
) {
|
) {
|
||||||
VideoClippingProperties properties = new VideoClippingProperties();
|
VideoClippingProperties properties = new VideoClippingProperties();
|
||||||
properties.getFolderScheduler().setInputDirectory(inputDirectory.toString());
|
properties.getFolderScheduler().setInputDirectory(inputDirectory.toString());
|
||||||
|
properties.getFolderScheduler().setPreserveInputQuality(preserveInputQuality);
|
||||||
Path workingDirectory = tempDir.resolve("working");
|
Path workingDirectory = tempDir.resolve("working");
|
||||||
if (!Files.exists(workingDirectory)) {
|
if (!Files.exists(workingDirectory)) {
|
||||||
try {
|
try {
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
video-clipping.folder-scheduler.enabled=false
|
||||||
Loading…
Reference in New Issue