How I Architected a Serverless AI Tool for Podcasts

Description and use

A serverless edge architecture on Cloudflare Workers was built to generate podcast show notes and titles, eliminating timeouts and delivering structured output.

Problem

Post‑production for podcasts required hours of manual work to produce show notes, titles, and SEO keywords, and existing AI wrappers caused latency, timeouts, and inconsistent outputs.

Outcome

Edge inference eliminated timeouts, produced consistent Markdown show notes, and enabled reliable title generation with tone control.

Workflow

• 1. Receive transcript JSON via Cloudflare Worker
• 2. Call @cf/meta/llama‑3‑8b‑instruct with strict system prompt for show notes
• 3. Return structured JSON response
• 4. For titles, build a separate prompt chain with tone and topic parameters
• 5. If transcript >4000 tokens, split into chunks, summarize each, then aggregate for final notes

Evidence

I was struggling with the post‑production grind. After recording a 45‑minute episode, the real work began: writing show notes, brainstorming titles, and pulling out SEO keywords.

Source

The payloads were huge, the functions were timing out, and the outputs were inconsistent.

Source

I chose Cloudflare Workers for the compute layer, specifically leveraging their AI inference capabilities at the edge.

Source

Edge AI is fast, but context windows are still a physical constraint. For episodes longer than 4,000 tokens, I had to implement a map‑reduce chunking strategy.

Source