The agent did exactly what I asked. That was the bug.

Описание и применение

Агент LLM выполнял запросы точно, но из‑за неверного плана выдавал неправильные результаты. Автор выявил, что автоматические хуки не могут обнаружить такие ошибки, и предложил четыре меры: уточнять решения, проверять первоисточник, сообщать предположения и использовать независимые артефакты проверки.

Задача

An LLM agent executed user requests correctly but produced incorrect results because the plan itself was flawed, and no tool call could detect the error.

Результат

By shifting the focus from executing the user’s request to validating the request’s correctness, the author reduced the occurrence of plan-level failures and increased the reliability of agent-generated outputs.

Рабочий процесс

• 1. Identify a task where the agent might follow a flawed plan. 2. Observe the agent’s correct execution but incorrect outcome. 3. Analyze why automated hooks cannot catch the error. 4. Implement four mitigations: decision-based prompts, primary source verification, assumption reporting, and independent artifact checks. 5. Apply mitigations to future tasks and monitor for improved error detection.

Доказательства

The agent is a premise-taker. It will interrogate its own implementation all day and will not touch the frame of the request, because the frame came from you and you're the one it's trying to satisfy.

Источник

I asked for a marketing asset. What found the bug was that a GIF is looked at by a human with eyes, and every assertion I had written shared an assumption with the code it was testing.

Источник

Four things I changed, in rough order of how much they've paid off.

Источник