The CLI backend reloads the ~3GB model per frame (~2min/frame). Add a SERVER mode: when LLAMACPP_SERVER_BIN is set, start llama-server once, POST base64 frames to its OpenAI /v1/chat/completions endpoint on loopback, stop it at the end. Same answers, ~3x faster on a full clip (measured: 3 frames 181s incl. one-time load vs ~6-8min). CLI mode (LLAMACPP_MTMD_BIN) remains the simple fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| beat_detect.py | ||
| local_asset_requirements.txt | ||
| local_asset_worker.py | ||
| local_cv_requirements.txt | ||
| local_cv_worker.py | ||
| provision_local_models.py | ||
| run_local_asset_worker.sh | ||
| run_local_cv_worker.sh | ||
| subject_track.py | ||
| vision_caption.py | ||
| vision_caption_llamacpp.py | ||