Feverdream: Generating Retro CGI Video with LLM and POV-Ray

Описание и применение

Проект Feverdream использует 3-миллиардную языковую модель и трассировщик лучей POV-Ray для создания видео в стиле 90-х. В отличие от диффузионных моделей, этот метод обеспечивает идеальную временную согласованность, точный контроль и низкую стоимость вычислений на потребительском оборудовании.

Задача

AI video diffusion models are expensive, computationally heavy, inconsistent, and hard to control, often resulting in geometric hallucinations and temporal incoherence.

Результат

The workflow produces coherent, period-accurate CGI video with perfect temporal coherence and exact control, running on consumer hardware at negligible cost compared to diffusion models.

Рабочий процесс

• Define a plain-English prompt describing the desired scene (e.g., 'chrome dolphin over a neon fractal canyon').
• Use ai_scene.py to send the prompt to a local LLM (default: qwen2.5-coder-3b-q4.gguf) via an OpenAI-compatible endpoint.
• The LLM generates a POV-Ray Scene Description Language (SDL) file, constrained by a system prompt to use specific 'retro90s.inc' macros for authentic 90s aesthetics.
• Run the sanitize() function to clean the LLM output, removing markdown fences, leading prose, and fixing common syntax errors like nested textures or incorrect color scaling.
• Render the sanitized SDL file using POV-Ray, a deterministic raytracer, to produce the final video frames.
• Compile the rendered frames into a video file.

Доказательства

A small LLM (a 3B coder model, running at ~95 tokens/sec on a consumer GPU) writes a POV-Ray scene description... Then POV-Ray, a deterministic raytracer with a 30-year lineage, renders it.

Источник

Temporal coherence is perfect. There's one real 3D scene. Objects don't morph between frames because they can't — they're geometric primitives with fixed coordinates.

Источник