The Limits of AI Verification: Intent vs. Specification in Developer Workflows

说明与用途

一位开发者探讨了AI在编程中的局限性。AI擅长根据规范验证代码，但无法理解开发者的意图和设计决策。主要风险并非被取代，而是人类因AI看似可靠而变得自满并放弃验证。人类必须保持控制并定义“正确”的含义。

问题

Developers face the risk of losing professional relevance as AI automates coding and verification tasks, potentially leading to a state where human oversight is abandoned due to convenience rather than necessity.

结果

The experiment revealed that AI excels at verifying code against specifications but fails to understand intent, design decisions, or the practical utility of a feature. The human's unique value lies in defining intent and maintaining rigorous oversight to prevent complacency.

工作流

• Deploy AI agents to perform initial coding, diagnosis, and verification passes.
• Allow agents to refute each other's diagnoses and stop execution on false premises.
• Review AI-generated defect lists to distinguish between actual bugs and intended design decisions (intent verification).
• Verify that code matches the specification, but critically assess if the specification matches the user's actual intent.
• In domains outside core expertise, refuse to sign off on AI conclusions without rigorous pressure-testing and exact alignment of dependencies.
• Maintain active control and checking habits to prevent 'human degradation' and complacency.

证据

A verifier can prove a system does what it says. It cannot tell you whether what it says is what you meant.

来源

The seat doesn't disappear because something takes it. It disappears because sitting in it stops feeling necessary.

来源

I removed three of them on sight — not because the analysis was wrong, but because the behaviour was intended.

来源