How I got an AI video model to change outfits exactly on the fan blade

Описание и применение

С помощью режима референса в Seedance 2.0 Mini удалось синхронизировать смену нарядов с проходом лопасти вентилятора, достигнув 100 % точности для до трёх нарядов.

Задача

The AI video model could not synchronize outfit changes with fan blade passes when only text prompts were used.

Результат

By providing a reference video that demonstrates the correct fan rhythm, the AI model could reliably synchronize outfit changes with blade passes for up to three outfits, achieving 100% accuracy in those cases.

Рабочий процесс

• Upload one photo per outfit to BlinkPop.
• Compose the scene with an overhead shot, carpet, sofa, and the first outfit image using an image model.
• Animate the stills into a short vertical clip with Seedance 2.0 Mini.
• For timing, create short rhythm clips from previous renders showing correct fan blade passes.
• Send the outfit stills as reference_image_urls and the rhythm clip as reference_video_urls to Seedance 2.0 Mini.
• Include a prompt that copies camera, fan, and timing from the reference video and uses the images for room, person, and outfits.
• Receive the rendered video and verify outfit changes align with blade passes.
• If the job fails, automatically refund credits and allow the job to proceed if the photo check service is down.

Доказательства

The first version gave the video model the stills and a prompt like "a ceiling fan turns slowly; each time a blade passes the lens, the outfit changes to the next one."

Источник

Seedance 2.0 Mini has a reference mode: besides reference images, you can pass a reference video, and it copies camera, motion and timing from it.

Источник

Results: 2 and 3 outfits: 8 out of 8 correct (man, woman and a dog)

Источник