How I Built a Reliable LLM Pipeline for Ad Creative Evaluation (with Strict Pydantic Contracts)

说明与用途

构建了本地LLM管道CreativeAudit，使用严格的Pydantic合同自动评估广告创意，首次审核时间从数十分钟降至秒级。

问题

Art directors spend 20–40 minutes per creative reviewing brand guidelines, mandatory elements, and forbidden techniques, making first‑pass review slow, expensive and inconsistent.

结果

First‑pass review time dropped from tens of minutes to seconds; output became consistent, machine‑readable, and critical brand violations were reliably caught.

工作流

• Collect brief and creatives
• Render Jinja2 prompt from brief and creatives
• Send prompt to local LLM (Ollama)
• Validate LLM output against strict Pydantic v2 schema
• If validation fails, reject and log error
• Score brand_alignment, constraint_compliance, message_clarity
• Compute total weighted score
• Derive verdict based on score and critical failures
• Return verdict, score, and explanation to UI or API

证据

First-pass review time dropped from tens of minutes to seconds

来源

The model is non-deterministic. I treat the Pydantic schema as a hard boundary: if the response doesn’t match the schema, it is rejected as an error — it never becomes a fake score.

来源

I wrote 38 unit tests. The external LLM is fully mocked, so tests are deterministic and run in under 2 seconds.

来源