The Limits of AI Verification: Intent vs. Specification in Developer Workflows

Описание и применение

Разработчик исследует границы возможностей ИИ в программировании. ИИ отлично проверяет код на соответствие спецификации, но не понимает намерения разработчика и дизайн-решения. Главная угроза — не замена человека, а человеческая лень и отказ от проверки из-за кажущейся надежности ИИ. Человек должен сохранять контроль и определять, что значит «правильно».

Задача

Developers face the risk of losing professional relevance as AI automates coding and verification tasks, potentially leading to a state where human oversight is abandoned due to convenience rather than necessity.

Результат

The experiment revealed that AI excels at verifying code against specifications but fails to understand intent, design decisions, or the practical utility of a feature. The human's unique value lies in defining intent and maintaining rigorous oversight to prevent complacency.

Рабочий процесс

• Deploy AI agents to perform initial coding, diagnosis, and verification passes.
• Allow agents to refute each other's diagnoses and stop execution on false premises.
• Review AI-generated defect lists to distinguish between actual bugs and intended design decisions (intent verification).
• Verify that code matches the specification, but critically assess if the specification matches the user's actual intent.
• In domains outside core expertise, refuse to sign off on AI conclusions without rigorous pressure-testing and exact alignment of dependencies.
• Maintain active control and checking habits to prevent 'human degradation' and complacency.

Доказательства

A verifier can prove a system does what it says. It cannot tell you whether what it says is what you meant.

Источник

The seat doesn't disappear because something takes it. It disappears because sitting in it stops feeling necessary.

Источник

I removed three of them on sight — not because the analysis was wrong, but because the behaviour was intended.

Источник