video_editing_poc/docs/input-folder-scheduler-plan.md

Input Folder Scheduler Plan

Goal

Add a scheduler that scans an input folder for video files and creates 8-second clips into an output folder.

Requested runtime paths:

• input: /input/source
• output: /output/clips

Repository-local placeholder directories:

• input/source
• output/clips

Note: the local machine's filesystem root is read-only in this workspace, so absolute /input/source and /output/clips could not be created here. In production, mount or create those absolute paths and configure the service to use them.

Milestones

1. Create repository-local placeholder directories input/source and output/clips.
2. Add folder scheduler configuration properties.
3. Create startup directory initialization for input, output, working, processed, and rejected folders.
4. Add deterministic scanner that picks one candidate video at a time.
5. Add repeat-prevention by moving selected inputs to a working folder before processing.
6. Add valid-video detection using extension filtering and ffprobe.
7. Add invalid-file handling that logs the reason and moves files to rejected storage.
8. Add FFmpeg clipping into /output/clips/<video-name>/ using 8-second MP4/H.264/AAC segments.
9. Add success handling that moves processed source files to processed storage.
10. Add failure handling for FFmpeg or filesystem errors.
11. Add unit tests for ordering, invalid-file handling, repeat prevention, and state transitions.
12. Add optional FFmpeg integration test for a short fixture.
13. Add local profile or documented runtime config for enabling the scheduler.
14. Run verification and mark feature complete.

Format Recommendation

Use .mp4 clips containing:

• video: H.264 / AVC via libx264
• audio: AAC via aac
• MIME type: video/mp4

Rationale:

• MDN documents MP4 as a popular, broadly supported container.
• MDN documents MPEG-4 / MP4 as supported by all major browsers and lists H.264 and AAC support for MP4.
• MDN specifically recommends MP4 with H.264 video and AAC audio as a broadly supported combination across every major browser.
• FFmpeg's segment muxer supports segment_time; its documentation notes that accurate splitting needs keyframes at the split points, so exact 8-second clips should force keyframes or transcode.

Sources:

• https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats/Containers
• https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats/Video_codecs
• https://ffmpeg.org/ffmpeg-formats.html#segment_002c-stream_005fsegment_002c-ssegment

Configuration

Add a new scheduler config group under video-clipping.folder-scheduler.

Recommended properties:

video-clipping.folder-scheduler.enabled=true
video-clipping.folder-scheduler.input-directory=/input/source
video-clipping.folder-scheduler.output-directory=/output/clips
video-clipping.folder-scheduler.processed-directory=/input/processed
video-clipping.folder-scheduler.rejected-directory=/input/rejected
video-clipping.folder-scheduler.working-directory=/input/working
video-clipping.folder-scheduler.poll-interval-ms=5000
video-clipping.folder-scheduler.segment-duration-seconds=8
video-clipping.folder-scheduler.ffmpeg-binary=ffmpeg
video-clipping.folder-scheduler.ffprobe-binary=ffprobe
video-clipping.folder-scheduler.output-container=mp4
video-clipping.folder-scheduler.exact-preset=veryfast

Keep the existing API-driven clipper config separate from this folder scheduler. The folder scheduler is a direct filesystem workflow, not an API/upload/session workflow.

Processing Rules

The scheduler should:

1. Create all configured directories on startup if they do not exist.
2. Scan input-directory on a fixed delay.
3. Ignore directories, hidden files, temporary files, and files currently being written.
4. Pick only one candidate at a time.
5. Process candidates in deterministic order, preferably filename natural order so 1.mp4 is processed before 2.mp4.
6. Validate a candidate with ffprobe, not only by file extension.
7. Move invalid files to rejected-directory and log why they were rejected.
8. Move a selected valid input to working-directory before processing, so the same file is not picked again while processing.
9. Write clips to output-directory/<video-name>/.
10. Move the source file to processed-directory only after all clips are successfully created.
11. If processing fails, move the source file to rejected-directory or leave it in working-directory with a failure marker, depending on operator preference.

Use the source filename stem for the output directory:

/input/source/1.mp4 -> /output/clips/1/clip_00000.mp4
/input/source/1.mp4 -> /output/clips/1/clip_00001.mp4
/input/source/1.mp4 -> /output/clips/1/clip_00002.mp4

If two input files share the same stem, append a stable suffix:

/output/clips/1
/output/clips/1-20260710T141500Z

Valid Video Detection

Use a two-stage validation:

1. Extension allowlist for cheap filtering:
   • .mp4
   • .mov
   • .m4v
   • .webm
   • .mkv
2. ffprobe verification for real validation.

Suggested ffprobe command:

ffprobe -v error \
  -select_streams v:0 \
  -show_entries stream=codec_type,codec_name,width,height \
  -show_entries format=duration \
  -of json \
  /input/working/1.mp4

Accept the file only if:

• ffprobe exits with code 0
• at least one video stream exists
• duration is present and greater than 0
• width and height are positive

Log invalid files with:

• source path
• reason
• ffprobe exit code
• sanitized ffprobe output

Do not log raw paths from untrusted metadata or full FFmpeg stderr if it contains sensitive filesystem details.

FFmpeg Command

Default to exact 8-second MP4/H.264/AAC output:

ffmpeg -hide_banner -y \
  -i /input/working/1.mp4 \
  -map 0 \
  -c:v libx264 \
  -preset veryfast \
  -crf 20 \
  -force_key_frames "expr:gte(t,n_forced*8)" \
  -c:a aac \
  -b:a 128k \
  -f segment \
  -segment_time 8 \
  -segment_format_options movflags=+faststart \
  -reset_timestamps 1 \
  /output/clips/1/clip_%05d.mp4

Expected behavior:

• each generated clip is approximately 8 seconds
• the final clip may be shorter than 8 seconds
• output files are broadly playable MP4 files

Implementation note:

• If exact boundaries are more important than speed, always transcode with libx264 and force keyframes.
• If speed is more important and keyframe-aligned cuts are acceptable, add an optional FAST mode using -c copy, but document that clips may not start exactly at 8-second boundaries.

Proposed Code Structure

Add:

• src/main/java/org/example/videoclips/folder/FolderSchedulerProperties.java
• src/main/java/org/example/videoclips/folder/FolderVideoScanScheduler.java
• src/main/java/org/example/videoclips/folder/FolderVideoValidator.java
• src/main/java/org/example/videoclips/folder/FolderFfmpegClipper.java
• src/test/java/org/example/videoclips/folder/FolderVideoScanSchedulerTest.java

Responsibilities:

• FolderSchedulerProperties: typed config for directories, binaries, poll interval, segment length, and output format.
• FolderVideoScanScheduler: scheduled scanner, single-file lock, ordering, state moves.
• FolderVideoValidator: extension filtering and ffprobe validation.
• FolderFfmpegClipper: FFmpeg command construction and process execution.
• Tests: validate ordering, invalid-file rejection, no repeat processing, and command construction.

Concurrency Control

Use an in-process guard to prevent overlapping scheduled runs:

• AtomicBoolean processing
• or ReentrantLock tryLock()

Only one file should process at a time. This directly satisfies:

1.mp4 completes before 2.mp4 starts

If the service will run multiple replicas against the same mounted folder, add a filesystem lock or database lease before enabling more than one scheduler instance.

File State Flow

Recommended state transitions:

/input/source/1.mp4
  -> /input/working/1.mp4
  -> /output/clips/1/clip_00000.mp4
  -> /output/clips/1/clip_00001.mp4
  -> /input/processed/1.mp4

Invalid file:

/input/source/not-video.txt
  -> /input/rejected/not-video.txt

Failed processing:

/input/working/broken.mp4
  -> /input/rejected/broken.mp4

This ensures the scheduler never processes the same source file repeatedly.

Operational Logging

Log these events:

• scheduler started and configured directories
• no candidate found at debug level
• selected candidate
• invalid candidate rejected
• FFmpeg command started, without sensitive metadata
• clip generation succeeded with clip count
• source moved to processed directory
• processing failed with sanitized error

Testing Plan

Unit tests:

• creates configured folders
• sorts 1.mp4 before 2.mp4
• ignores hidden and temporary files
• rejects invalid extension before ffprobe
• rejects ffprobe failure
• moves valid input to working before processing
• moves successful source to processed
• does not pick the same input twice

Integration tests:

• run FFmpeg against a short generated fixture if ffmpeg and ffprobe are installed
• verify a 17-second input produces 3 clips
• verify output path is output/clips/<video-name>/clip_%05d.mp4

Rollout Plan

1. Add config properties and disabled-by-default scheduler.
2. Add directory creation and scanning behavior.
3. Add validation and rejection flow.
4. Add FFmpeg clipping flow.
5. Add unit tests for file-state transitions.
6. Add optional integration test gated on local FFmpeg availability.
7. Enable in a local profile first.
8. Enable in production only with mounted /input and /output volumes.

Acceptance Criteria

The feature is complete when:

• /input/source and /output/clips are configurable and created on startup
• valid video files are processed one at a time
• 1.mp4 is processed before 2.mp4 under deterministic ordering
• clips are written to /output/clips/<video-name>/
• each clip is MP4/H.264/AAC by default
• invalid files are ignored for processing, moved to rejected storage, and logged
• processed source files are not picked up again
• tests cover ordering, invalid file handling, and repeat-prevention behavior