My agent looked fine — until I gave it an empty list

说明与用途

作者构建了一个小型工具，用于在10分钟内测试自己的AI代理，而不是信任它看起来正确。该工具对每个任务评分4项：完成、正确、没有捏造、没有错误操作。修复后的答案得分为4/4，原始未修复函数得分为3/4并丢失了“是否正确”得分。

问题

An AI agent can appear correct on a task but still fail when edge cases are introduced, so a quick automated check is needed to catch silent failures before trusting the agent's output.

结果

The fixed answer scored 4/4 (finished, correct, did not make things up, took no wrong action); the original unfixed function scored 3/4 and lost the 'Is correct' point.

工作流

• Paste your agent's answer into the tool at https://jefferson999.github.io/agent-reliability--score/
• Run the score and read what to fix
• If the agent passes the empty-list task, try the other 19 tasks
• Use the tool as a quick first check, not as a CI or production replacement

证据

One test from it: 'Handle an empty average.' The starting code divides by the list length, so an empty list breaks it.

A fixed answer — check for an empty list first and return 0, otherwise return the normal average — scored 4/4. What-to-fix: 'No fix needed for this task.'

The original, unfixed function scored 3/4. It lost the 'Is correct' point. The plain-language fix: tell the agent to check for an empty list first and return 0, and keep the normal average for everything else.

It scores 4 things per task: finished, correct, did not make things up, and took no wrong action. Auto-score suggests, you confirm — so you get speed without handing the judgment to a black box.