Why I don't automatically retry timed-out AI video requests

Description and use

When a video generation request times out, the system records a submission marker before calling the external API to prevent duplicate paid tasks and route uncertain jobs to manual review.

Problem

When a video generation request times out, retrying can unintentionally create duplicate paid tasks because the provider may already be rendering the original request.

Outcome

By recording a submission marker before external calls and distinguishing job states, the workflow avoids duplicate paid tasks, reduces unnecessary retries, and provides clear paths for manual review and credit restoration.

Workflow

• Record a submission timestamp before calling the generation API
• If no submission marker and no provider ID, claim the job, record the attempt, then submit
• If a provider ID exists, query that existing task
• If a submission marker exists but no provider ID, hold the job for support review
• If an output URL is already recorded, continue processing or saving that video
• For failed storage, record source video URL to resume from existing output
• Apply bounded retries to the soundtrack step and fall back to silent footage if exhausted
• Restore user credits on confirmed generation failure, reserve ambiguous submissions for investigation

Evidence

The database updates are conditional, and processing uses a short lease so overlapping workers don't casually pick up the same job.

Source

If the worker crashes after writing the marker but before sending the request, the job also ends up waiting for review.

Source

A provider-side idempotency key would be a better way to close that gap.

Source