How My Metadata Script Skips Bad Files Instead Of Crashing A 300-File Run

Description and use

The metadata script processed 300 files but crashed on a zero‑byte image, producing an incomplete CSV. By validating each file and wrapping processing in try/except, it now skips bad files, logs errors, and outputs a clean CSV, avoiding account flagging.

Problem

A zero‑byte JPG caused a metadata script to crash, producing an incomplete CSV and no error log.

Outcome

The updated script processes all valid files, logs errors, and produces a clean CSV, preventing account flagging due to partial uploads.

Workflow

• 1. Validate file existence, size, and extension before processing.
• 2. Wrap per‑file processing in try/except to catch errors.
• 3. On failure, write filename and reason to errors.log and continue.
• 4. After loop, output CSV with only valid rows and a summary of successes and failures.
• 5. Use upload_tracker.py to track processed files and avoid double‑posting.

Evidence

Around file 84 the script hit a zero-byte JPG that a render subprocess had left behind, threw on Image.open(), and died.

Source

Now every file goes through its own try/except, and a failure writes one line to errors.log with the filename and the reason, then moves on.

Source

The second version of the script wrote rows first and hoped. Now it checks each file before a row gets built: does the file exist, is it non-zero bytes, does the extension match what I passed with --ext, can the title be derived.

Source