How I added a local AI to my free video app (no API key, runs on your PC)

说明与用途

AI Video Studio 现在使用本地 Qwen3‑4B 模型和 llama.cpp 自动将普通文本转换为所需的脚本格式，消除了特殊语法和外部 API 密钥的需求。

问题

The app’s script‑writing required a custom format, making it tedious for users; the developer wanted the app to generate scripts from plain text without external API keys.

结果

The local AI integration allows the app to rewrite plain text into the required script format, reducing user effort and eliminating external API dependencies while maintaining reasonable performance on consumer hardware.

工作流

• 1. Bundle llama‑cpp’s llama‑server with the app.
• 2. Download the 2.5 GB Qwen3‑4B GGUF model once, verify SHA‑256, and store it outside the install folder.
• 3. On launch, start llama‑server as a separate process, send a single request, then terminate it to free memory.
• 4. Prompt the model with a JSON schema that specifies a fixed number of scenes (minItems=maxItems) and includes a worked example to enforce structure.
• 5. If the model fails (out of memory, slow PC, odd output), fall back to a plain sentence splitter and inform the user.
• 6. Optionally enable the Vulkan GPU build for faster inference (e.g., 3 s on RTX 4060 vs 23 s on CPU).

证据

I build AI Video Studio, a free and open‑source Windows app that turns a script into a short video: stock footage, a neural voice and word‑by‑word captions.

来源

The app starts llama‑server as a separate process, sends one request, and kills it straight after. That matters because rendering the video needs the memory back.

来源

Force the output with a JSON schema: a list of scenes, each with visual (search words) and voice (one spoken line).

来源

On an RTX 4060, the same script takes about 3 seconds on the card and about 23 seconds on the processor.

来源