How I added a local AI to my free video app (no API key, runs on your PC)

Description and use

AI Video Studio now employs a local Qwen3‑4B model via llama.cpp to automatically rewrite plain text into the required script format, removing the need for special syntax and external API keys.

Problem

The app’s script‑writing required a custom format, making it tedious for users; the developer wanted the app to generate scripts from plain text without external API keys.

Outcome

The local AI integration allows the app to rewrite plain text into the required script format, reducing user effort and eliminating external API dependencies while maintaining reasonable performance on consumer hardware.

Workflow

• 1. Bundle llama‑cpp’s llama‑server with the app.
• 2. Download the 2.5 GB Qwen3‑4B GGUF model once, verify SHA‑256, and store it outside the install folder.
• 3. On launch, start llama‑server as a separate process, send a single request, then terminate it to free memory.
• 4. Prompt the model with a JSON schema that specifies a fixed number of scenes (minItems=maxItems) and includes a worked example to enforce structure.
• 5. If the model fails (out of memory, slow PC, odd output), fall back to a plain sentence splitter and inform the user.
• 6. Optionally enable the Vulkan GPU build for faster inference (e.g., 3 s on RTX 4060 vs 23 s on CPU).

Evidence

I build AI Video Studio, a free and open‑source Windows app that turns a script into a short video: stock footage, a neural voice and word‑by‑word captions.

Source

The app starts llama‑server as a separate process, sends one request, and kills it straight after. That matters because rendering the video needs the memory back.

Source

Force the output with a JSON schema: a list of scenes, each with visual (search words) and voice (one spoken line).

Source

On an RTX 4060, the same script takes about 3 seconds on the card and about 23 seconds on the processor.

Source