We ran 150 real AI agent outputs through COGEXT. Here’s what we found.

说明与用途

对150个真实AI代理输出的审计显示，只有0.8％包含可验证的承诺；大多数承诺缺少收件人和截止日期，导致即使置信度高也无法验证。

问题

AI agents often make promises but there is no system to track whether those promises are kept, and many promises lack verifiable details such as recipients and deadlines.

结果

Out of 120 unbiased samples, only 1 commitment was found (0.8%). In a promise‑enriched set of 30 samples, 13 commitments were extracted; 61.5% lacked recipients and 92.3% lacked deadlines, rendering them unverifiable despite high confidence. The audit demonstrates that many agent promises are incomplete and cannot be automatically tracked.

工作流

• Collect 150 agent outputs from public sources (GitHub notebooks, DEV.to articles, Hacker News comments).
• POST each sample to COGEXT /api/v1/ingest with 1‑second delay.
• Analyze extracted commitments: count, recipient presence, deadline presence, classification, confidence, verifiability status.
• Generate verifier queries for each commitment.
• Compile findings and publish audit results.
• Provide dataset and scripts for reproducibility.
• Invite users to run audit on their own agents.

证据

Out of 120 randomly collected agent outputs, COGEXT extracted 1 commitment.

来源

61.5% had no recipient named.

来源

92.3% had no deadline.

来源