Every AI email tool has the same bug. It is not a bug in the model

Описание и применение

В большинстве AI‑почтовых инструментов модель получает власть принимать решения, но ограничивается инструкцией в подсказке, что приводит к уязвимости. Предложена архитектура, где модель оценивает четыре признака, а правила решают действие, а одобрение связывается с байтовым отпечатком, обеспечивая безопасность.

Задача

AI email tools give the model authority to decide actions, but rely on instructions in the prompt to constrain it, creating a security flaw where malicious emails can manipulate the model into taking irreversible actions without human approval.

Результат

The proposed architecture reduces reliance on expensive LLMs, improves auditability, and ensures irreversible actions are only taken after byte‑level approval, mitigating malicious manipulation.

Рабочий процесс

• 1. Receive inbound email
• 2. Model parses and scores four features
• 3. Deterministic rule engine maps scores to one of five lanes (interrupt, scheduling, queue, file, never render)
• 4. If lane allows unattended action, a separate flag checks account‑level mode
• 5. For irreversible actions, generate a sha256 receipt of the canonical payload
• 6. User approves by confirming the receipt
• 7. At execution, hash payload again and compare to receipt; if match, perform action, else abort

Доказательства

In my own tool the model never picks the outcome. It scores four features per email — confidence, sender trust, reversibility, urgency — and a deterministic, unit‑tested rule maps those four numbers to exactly one of five lanes

Источник

Approval has to bind bytes, not intent. So: approving mints a receipt — a sha256 over the exact canonical payload bytes at the moment you click yes.

Источник