Requirements.txt as Discipline: Stdlib-Only Scripts Need No Install

Description and use

Splitting the pipeline into small scripts that rely only on Python’s standard library eliminated double‑posting and allowed the entire workflow to run on a 2 GB server without installing dependencies.

Problem

Running a batch twice caused a double‑post flag on a stock platform, despite no crashes or errors.

Outcome

The refactored pipeline produced 39 finished products (856 MB) on a single‑core, 2 GB server without installing any packages, and eliminated double‑post flags.

Workflow

• Split the original pipeline into four scripts: make_metadata.py, upload_tracker.py, md2pdf.py, pack_product.py.
• Make make_metadata.py and upload_tracker.py depend only on the standard library.
• Use upload_tracker.py to record posted assets and prevent re‑runs from double‑posting.
• Run make_metadata.py with arguments like "python make_metadata.py ./my-assets --platform adobe --out metadata.csv --ext .jpg" to generate platform‑specific CSV headers.
• Use md2pdf.py and pack_product.py for rendering, installing only Pillow and reportlab as needed.
• Follow resource limits: single process, gc.collect() after each asset, measure peak RSS, run heavy renders in separate subprocesses, inspect previews at 60 dpi.
• Maintain a fixed folder layout for assets, output, packs, research, and scripts.
• Keep the research folder permanently for future product creation.
• Deploy the scripts on any machine with Python 3.9+ without creating a virtual environment or installing dependencies.

Evidence

The second time I ran my upload batch, I got the email I had been dreading. A channel had flagged my account for double-posting.

Source

I split my pipeline into small standalone scripts, I made two of them depend on nothing but the Python standard library, and I wrote a tracker so a re‑run could never double-post again.

Source

That split is the whole point. A CSV generator and a ledger are pure text and logic. They have no business pulling in an image library, and when they do not, you can run them anywhere Python exists.

Source

On a 1 vCPU, 2 GB box with no GPU, the pipeline only works if you respect those limits: One process at a time. Never fan out image jobs in parallel on a single-core box.

Source