I Let an LLM Write Cold Emails but Never Decide Who Gets Them: Lessons From 678 Real Sends

Description and use

An automated cold‑email pipeline using an LLM was redesigned so that the LLM only generates prose, while all decisions about recipients, sending, and reply classification are handled by deterministic, unit‑tested functions. This reduced mis‑targeting, prevented product hallucinations, and limited repeated outreach to the same company.

Problem

Automated cold‑email pipelines can send messages to the wrong recipients, invent product details, or repeatedly contact the same company, causing reputational damage and wasted effort.

Outcome

The architecture reduced accidental mis‑targeting, prevented the LLM from inventing product details, and introduced robust reply classification, resulting in 678 real sends with controlled outreach and 129 second approaches held back.

Workflow

• 1. Pull leads from Apollo.io with filtered industry names and closed‑set seniority/size values.
• 2. Enrich leads via Apollo person‑match, applying the acceptEnrichment guard to ensure correct email, domain, and surname match.
• 3. Generate email body with the LLM, constrained to approved template content and recipient website text.
• 4. Post‑process the LLM output: strip added sign‑offs, restore presentation links, and translate while preserving brand tokens.
• 5. Store a frozen copy of the final email to prevent silent template changes.
• 6. Schedule sending: one message per tick with ±50% jitter within a 07:00‑19:00 window.
• 7. Send via Microsoft Graph, then read replies and classify them using a deterministic rule set (bounce, auto‑reply, opt‑out, reply).
• 8. Update ownership and company stamps to prevent duplicate outreach.
• 9. Log results and human‑readable reasons for each classification.

Evidence

Connecting a contact database, an LLM and a company mail server is the easy part. A weekend gets you a pipeline that pulls a thousand leads, has GPT write to each one, and sends the result unattended.

Source

The model writes prose and picks from lists. It never decides whether, when, or to whom a message is sent.

Source

The identity guard: function acceptEnrichment(...) { … }

Source

Result: 129 second approaches held back in nine weeks.

Source