My agent looked fine — until I gave it an empty list

Description and use

The author built a small tool to test their own AI agent in 10 minutes instead of trusting that it 'looked right.' The tool scores 4 things per task: finished, correct, did not make things up, and took no wrong action. A fixed answer scored 4/4; the original unfixed function scored 3/4 and lost the 'Is correct' point.

Problem

An AI agent can appear correct on a task but still fail when edge cases are introduced, so a quick automated check is needed to catch silent failures before trusting the agent's output.

Outcome

The fixed answer scored 4/4 (finished, correct, did not make things up, took no wrong action); the original unfixed function scored 3/4 and lost the 'Is correct' point.

Workflow

• Paste your agent's answer into the tool at https://jefferson999.github.io/agent-reliability--score/
• Run the score and read what to fix
• If the agent passes the empty-list task, try the other 19 tasks
• Use the tool as a quick first check, not as a CI or production replacement

Evidence

One test from it: 'Handle an empty average.' The starting code divides by the list length, so an empty list breaks it.

A fixed answer — check for an empty list first and return 0, otherwise return the normal average — scored 4/4. What-to-fix: 'No fix needed for this task.'

The original, unfixed function scored 3/4. It lost the 'Is correct' point. The plain-language fix: tell the agent to check for an empty list first and return 0, and keep the normal average for everything else.

It scores 4 things per task: finished, correct, did not make things up, and took no wrong action. Auto-score suggests, you confirm — so you get speed without handing the judgment to a black box.