Jev vs Claude for OpenPoke: Email Screening Speed and Cost

说明与用途

在 OpenPoke 中使用 Jev 可将邮件筛选速度提升约5.8倍，成本降低至 Claude Sonnet 的1/95，且保持高一致性。

问题

Improve email triage speed and reduce cost in the OpenPoke assistant by replacing the default chat model with Jev for the screening decision.

结果

Using Jev for the screening decision in OpenPoke reduced mean latency from 2,452 ms to 424 ms and cut input cost from $4.63 to $0.049 per 1,000 screens, while maintaining high agreement with Claude Sonnet. The injection gate achieved a 95.3 % trigger rate against hostile text, preventing low‑priority messages from surfacing.

工作流

• 1. Fork OpenPoke and add Jev integration. 2. Replace the single boolean call with a Jev call that asks four questions: importance, security‑code content, bulk mail, and AI‑assistant instructions. 3. Route decisions: if Jev deems mail unimportant, skip the LLM; if important, proceed to summarisation; if uncertain, fall back to the original classifier. 4. Add Jev checks before tool execution and for search relevance. 5. Run evaluation harnesses on 12 synthetic emails, measuring latency and cost for both Claude Sonnet and Jev. 6. Perform a prompt‑injection test with hostile text to validate the injection gate. 7. Deploy the fork via the GitHub repo and enable the TYPESAFE_API_KEY for live operation.

证据

Mean screening latency went from 2,452 ms with Claude Sonnet to 424 ms with Jev.

来源

Estimated input cost went from $4.63 to $0.049 per 1,000 screens.

来源

The injection gate fired in 95.3% of attack trials across 12 synthetic carriers.

来源