How I added a local AI to my free video app (no API key, runs on your PC)

Описание и применение

Приложение AI Video Studio теперь использует локальную модель Qwen3‑4B, запущенную через llama.cpp, чтобы автоматически преобразовывать обычный текст в формат сценария, избавляя пользователей от необходимости вводить специальные команды и от внешних API‑ключей.

Задача

The app’s script‑writing required a custom format, making it tedious for users; the developer wanted the app to generate scripts from plain text without external API keys.

Результат

The local AI integration allows the app to rewrite plain text into the required script format, reducing user effort and eliminating external API dependencies while maintaining reasonable performance on consumer hardware.

Рабочий процесс

• 1. Bundle llama‑cpp’s llama‑server with the app.
• 2. Download the 2.5 GB Qwen3‑4B GGUF model once, verify SHA‑256, and store it outside the install folder.
• 3. On launch, start llama‑server as a separate process, send a single request, then terminate it to free memory.
• 4. Prompt the model with a JSON schema that specifies a fixed number of scenes (minItems=maxItems) and includes a worked example to enforce structure.
• 5. If the model fails (out of memory, slow PC, odd output), fall back to a plain sentence splitter and inform the user.
• 6. Optionally enable the Vulkan GPU build for faster inference (e.g., 3 s on RTX 4060 vs 23 s on CPU).

Доказательства

I build AI Video Studio, a free and open‑source Windows app that turns a script into a short video: stock footage, a neural voice and word‑by‑word captions.

Источник

The app starts llama‑server as a separate process, sends one request, and kills it straight after. That matters because rendering the video needs the memory back.

Источник

Force the output with a JSON schema: a list of scenes, each with visual (search words) and voice (one spoken line).

Источник

On an RTX 4060, the same script takes about 3 seconds on the card and about 23 seconds on the processor.

Источник